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Abstract 

This exploratory study evaluated self-regulated learning 
(SRL) strategies of 27 students in grades 9-12 during an 
engineering design project. The specific focus of the study 
was on student task interpretation and its relation to 
planning and cognitive strategies in design activities. Two 
research questions guided the study: (1) To what degree 
was students' task interpretation reflected in their working 
plans and SRL strategies use across the design process?; 
and (2) How did relatively lower- and higher-achieving 
design-performing students differ in interpreting tasks and 
deploying SRL strategies? Survey instruments and Web- 
based design notebook writing were used to capture 
students' reported use of SRL strategies. On the survey, 
students reported use of SRL strategies at the early, 
middle, and final stages of the design task, with a focus on 
task interpretation, planning, cognitive, monitoring/fix-up 
strategies, and performance criteria. 

The findings suggested that students scored higher on task 
interpretation than on planning, cognitive, and 
monitoring/fix-up strategies. Students' relatively high 
awareness of task interpretation-related-issues was also 
reflected in what they considered to be good design 
performance. Our findings were suggestive that higher 
performing students scored significantly higher than their 
lower-performing peer on cognitive and monitoring and 
fix-up strategies. On the other hand, lower-performing 
students reported greater use of planning strategies. 
Moreover, higher-performing students seemed to be able 
to convey more detailed and specific descriptions than did 
their relatively-lower performing peers. This article 
discusses potential implications for design instruction in 
grades 9-12. 
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Introduction 

There is a widely accepted theory among Science, 
Technology, Engineering, and Mathematics (STEM) 
educators that teaching concepts and skills in the context 


of solving engineering design-type problems will enable 
students to more easily grasp and retain those concepts 
(Katehi et al, 2009). Although there is a strong push 
among educators to emphasise engineering design in 
K-12 engineering education either in Australia, Europe, or 
United States (Eder, 1993; Gol et al, 2004; Katehi et al, 
2009), few studies have assessed how students' self- 
regulated learning (SRL) strategies are selected and 
deployed to solve complex problems (De Corte et al, 
2011). Having a better understanding about those 
strategies is essential, however, because consistency in 
employing SRL strategies is highly correlated with student 
achievement (Zimmerman and Schunk, 2011), 
particularly in the kinds of complex, ill-structured tasks so 
ubiquitous in Engineering education and practice. 

SRL can be defined essentially as a form of iterative, goal- 
directed activity that involves interpreting tasks, setting 
goals, selecting, adapting or even inventing strategies 
effective for achieving those goals, monitoring progress, 
and adjusting approaches as needed (Zimmerman, 

2008). Effective SRL is invited by and particularly critical in 
complex or ill-structured tasks, such as engineering design 
(Lawanto, 2010; Lawanto and Johnson, 2012). As stated 
by Dym and Little, "Designing is about people planning 
and creating ways to produce things that achieve some 
known goals" (2009: 5). The design process is not linear; 
at any phase of the process, students must identify, plan, 
act, evaluate, and make necessary adjustments. Students' 
development of SRL strategies is particularly essential 
when working on ill-structured problems such as 
engineering design tasks that are more difficult to solve 
and require more cognitive operations than do well- 
structured ones (Paris and Winograd, 1990). 

As an illustration, when working on a design project such 
as designing and building a robot that can move soda 
cans from one place to another or a new library that has a 
full-service facility, students are expected to correctly 
interpret task requirements before formulating strategic 
plans to achieve them. These plans are expected to 
identify focused cognitive strategies necessary to project 
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completion. Students are then expected to enact those 
strategies, monitor progress, and adjust approaches in 
order to achieve important criteria as suggested by design 
problems. In this research, we examined how students 
engaged in these cycles of SRL in the context of these two 
design activities. 

More specifically, Butler (1998) found that having a good 
understanding of a presented learning activity grounded in 
productive metacognitive knowledge about tasks is 
essential to students' successful planning, self-monitoring, 
and selection of appropriate strategies to accomplish task 
objectives (see also Butler and Cartier, 2004b). Thus, in 
this research, we focused specifically on evaluating how 
students' understanding about a design task might be 
reflected in planning, cognitive actions, and monitoring 
and fix-up strategies to solve and enhance design 
processes and outcomes. 

Relevant literature review 

Insights into metacognition in a self-regulated learning 
context 

What is metacognition? In simple language, metacognition 
refers to thinking about thinking (Brown, 1987). While the 
term cognition refers to one's ability to build knowledge and 
involves information processing, knowledge acquisition, and 
problem solving, the term metacognition refers to both 
understanding about and control of that working cognition 
(Flavell, 1979; Gourgey, 1998). Scholars define 
metacognition in numerous ways, many of which overlap; 
however, most definitions identify as metacognitive as 
comprising two main components: (1) metacognitive 
knowledge and (2) metacognitive control. For example, 
informed by the classical theories of metacognition 
introduced by Flavell (1976), Pintrich (2002) described 
metacognition as involving both knowledge and control. 
Students hold metacognitive knowledge about strategies 
that might be used for a particular task and the conditions 
under which the strategies might be useful. Metacognitive 
control encompasses processes that learners use to 
monitor and self-regulate cognition and learning. 

While many theoretical perspectives on metacognition and 
self-regulation have been offered (e.g., Butler and Winne, 

1995; Winne and Fladwin, 1998; Zimmerman, 2002; 
Zimmerman and Schunk, 2001), for this research we chose 
to build from Butler and Cartier's socio-constructivist model 
of self-regulation (Butler and Cartier, 2004a; Butler and 
Cartier, 2005; Cartier and Butler, 2004) because it enables 
teasing apart and investigating the interplay between 
metacognitive knowledge (e.g., students' understandings 
about tasks and strategies, as mediating variables), and 
metacognitive control, conceptualised as cycles of "self¬ 


regulation in action," within the context of complex learning 
activity. This model depicts eight central features that 
interact with each other to shape engagement in learning: 
layers of context, what individuals bring, mediating variables, 
task interpretation, personal objectives, SRL processes, 
cognitive strategies, and performance criteria. 

First, layers of context refer to the learning environments, 
such as school, classroom, and instructional approaches, in 
which students engage in learning. In this project we attend 
to contextual influences by recognising how our study of 
SRL was situated in a particular kind of activity, during which 
students were engaged in an engineering design project 
together in particular projects and classrooms. Second, 
students bring to those contexts strengths, challenges, and 
interests that shape their engagement (e.g., by influencing 
whether they perceive activities to be difficult/easy or 
boring/important). Third, examples of variables that mediate 
student engagement in a learning task include students' 
conceptions of academic work, derived through prior 
experience (e.g., about learning in STEM as about 
memorising facts from textbooks), metacognitive 
knowledge (e.g., about tasks, strategies, learning), self¬ 
perceptions (e.g., of competence and control over 
outcomes) and emotions. In this project, because of our 
more targeted focus, we did not assess directly individuals' 
histories or a full set of potential mediating variables, as has 
been conducted in other research (e.g., Butler et al, 2011). 
However, through our attention to task interpretation, we 
created the opportunity to infer how students' prior 
experiences may have influenced their conceptions about 
learning in engineering, and the metacognitive knowledge 
they may have constructed related to design tasks. 

Our focus in this project is more specifically on the fourth 
component of the Butler and Cartier model, task 
interpretation, and its relationship to students' use of 
cognitive and self-regulating strategies. Task interpretation is 
the heart of the SRL model inasmuch as it shapes key 
dynamic and recursive self-regulating processes (Butler and 
Winne, 1995). Students' interpretation of task demands is a 
key determinant of the goals set while learning, the 
strategies selected to achieve those goals, and the criteria 
used to self-assess and evaluate outcomes. When students 
self-regulate learning effectively, task interpretation and 
personal objectives, the fifth feature of the Butler and 
Cartier model, work together to inspire students' activation 
of effective self-regulating and cognitive strategies during a 
design task. 

Sixth, in light of their task interpretations and personal 
objectives, students manage engagement in academic 
work by using a variety of self-regulating strategies: 
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planning, monitoring, evaluating, and adjusting approaches 
to learning. Ideally students plan how to use available 
resources, select strategies for task completion, self¬ 
monitor progress, and adjust goals, plans, or strategies 
based upon self-perceptions of progress or feedback and 
performance. These strategies are iterative and dynamic 
endeavors. 

A seventh feature of the Butler and Cartier model is that 
they situate students' use of cognitive strategies in the 
context of cycles of dynamic, iterative self-regulating 
activities. Students with good metacognitive skills and 
awareness use these processes to oversee their learning 
process (e.g., to plan and monitor ongoing cognitive 
activities). Finally, as part of that monitoring effort and, in 
an on-going way, students compare outcomes that 
emerge through their activity (process, cognitive) with 
internal or external standards (Flavell, 1979). Linked to 
task interpretation and personal goals, performance criteria 
are the standards against which students judge the 
progress they are making (see also Butler and 
Winne, 1995). 

Zimmerman and Pons (1986) found that consistency in 
employing self-regulated learning strategies is highly 
correlated with student achievement. Schoenfeld (1983) 
argued that lack of success in problem-solving may result 
from the absence of assessments and strategic decisions. 
Thus, students with poor metacognition (knowledge and 
control) may benefit from training to improve 
metacognition and subsequent learning performance 
(Coutinho, 2008). When students engage in complex and 
ill-structured problem-solving (as in design tasks), these 
SRL features dynamically interact. Thus, in this study our 
goal was to document the dynamic interconnections 
between students' interpretation of design tasks, self¬ 
regulating and cognitive strategies, and criteria students 
associate with success. 

Note that, while models of "self-regulation focus on how 
individuals engage in iterative cycles of adaptive learning 
activity, many researchers also describe how self-regulation 
is mediated socially (e.g., Bandura, 2006; Vygotsky, 

1978). Thus, when peers learn together, as is frequently 
the case within design projects, forms of co-regulation 
may emerge, where students shape and scaffold each 
other's engagement (e.g., see Hadwin and Jarvela, 2011; 
Volet, Summers, and Thurman, 2009). In this research, an 
important contextual influence was that students were 
engaged in design activities together. 

Engineering design process 

Design problems are among the most complex and ill- 


structured tasks encountered in engineering practice 
(Jonassen, 2000; Reitman, 1965; Simon, 1973). For 
many years, researchers (Reitman, 1965; Simon, 1973) 
have characterised design problems as ill-structured 
because they have ambiguous specification of goals, no 
determined solution path, and require the integration of 
multiple knowledge domains. Goel and Pirolli (1989) 
articulated the characteristics of design problems, including 
many degrees of freedom in the problem statement, 
which consists only of goals and intentions, limited or 
delayed feedback from the world, artifacts as outputs that 
must function independently of the designer, and 
"answers that tend to be neither right nor wrong, only 
better or worse" (Jonassen, 2000: 80). 

Solving an engineering design problem is a structured and 
staged process. In a model similar to those proposed by 
Christiaans (1992) and Cross (2000), Dym and Little 
(2009) contend that the design process consists of five 
phases: problem definition, conceptual design, preliminary 
design, detailed design, and design communication. These 
design phases are considered high-level overall views of 
design processes. They involve a sequence of actions or 
design strategies, which are self-contained cognitive 
approaches and relate to the current state of the 
design process. 

In this study, we focused on the first two of Dym and 
Little's five design phases: problem definition and 
conceptual design. These two phases were selected 
because students' success in understanding the objectives 
of the project and how to conceptually solve a design 
problem has a significant impact on the remaining three 
design phases. Dym and Little (2009) divided these two 
phases into several sub-phases that we draw on in our 
analyses. The problem definition phase consists of four 
sub-phases: clarifying objectives, establishing metrics for 
objectives, identifying constraints, and revising a client's 
problem statement. In this study, students were working 
on an assigned task as part of the course requirement, 
with no option to change or revise the task. As a result, we 
attended only to the first three of these problem definition 
sub-phases. The second phase, conceptual design, 
involves six sub-phases: establishing functions, defining 
requirements, establishing means for functions, generating 
design alternatives, refining and applying metrics to design 
alternatives, and choosing a design solution. 

Findings from previous studies suggested that 
metacognitive skills are essential in solving engineering 
design project because of the nature and complexity of 
the design processes (e.g., Lawanto, 2010; Lawanto and 
Johnson, 2012). The ways in which students use 
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strategies, observe what transpires, and search for 
alternative solutions are examples of how metacognition is 
applied during these phases of the design process. 

The Study 
Overview 

A central goal of this study was to describe the task 
interpretation of students engaged in a design activity and 
determine the extent to which they translated their 
understanding of the design task into planning and 
cognitive strategies. Two research questions guided this 
exploratory study: (1) To what degree was students' 
interpretation of the design task reflected in working plans 
and selected cognitive strategies across design process?; 
and (2) How did lower- and higher-design-performing 
students differ in interpreting tasks and deploying SRL 
strategies? 

Study participants 

A purposeful sampling strategy (Creswell, 2003) was 
employed for this study. One school in Colorado was 
selected because of its comprehensive set of 
engineering/technology-related courses. All students 
enrolled in both Architectural and Robotics Designs classes 
were invited to participate. Twenty-nine students, age 
ranged between 15 and 18, participated in the study: 
seven students (five females and two males) were in the 
Architectural Design class and 22 students (three females 
and 19 males) were in the Robotics Design class. 

The context of the design tasks 

Students enrolled in both design classes met four times a 
week, taught by the same teacher, and none of the 
students was enrolled in both classes. They were given a 
design task to work on as their final design project. They 
had three weeks to complete their project. Grades were 
assigned for their project and were counted towards each 


student's overall course grade. Students' grades used in 
this study reflected individual design performance. The 
requirements of the design projects were created by the 
teacher. A brief description of these two final projects is 
presented below. 

Robotic design 

Students were required to work in a team of two or three 
to design and build a robot capable of operating under a 
tele-operated mode, inside a 122 cm x 244 cm table with 
5 cm high walls populated with eight empty soda cans. 

The cans were painted and divided into two colours, red 
and blue. Four of the cans (two blue and two red) sat atop 
pedestals, 15 cm high, positioned in each corner. Students 
were then required to swap the positions of the cans. Cans 
sitting on the pedestals were lowered and replaced with a 
can which had been sitting on the table surface. Students 
were given two and a half minutes to complete as many 
exchanges as possible. Students were required to fully 
design their robots, including drive train and actuator, using 
SolidWorks™ before building and competing in four 
matches. Grading criteria took into account both group and 
individual performance, and were established based upon 
the SolidWorks™ model, robot performance, team 
participation, and design journal writings. 

Architectural design 

Architectural design students designed and built a 
miniature of a new library to be built in a small town with 
a population of 25,000. As in the robotic design project, 
the architectural design students were required to 
undertake this task as their final design project. The library 
was to be built on a square corner lot measuring 150' x 
150,' needed to include various facilities such as meeting 
rooms, performance space, a computer access area, an 
outside garden or reflection area with benches, a 
circulation desk, a staff office or break space, and 
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Figure 1. Activity and design artifact example in robotics design class 


Design and Technology Education: An International Journal 18.1 


47 


RESEARCH 






RESEARCH 


An Exploratory Study of Self-Regulated Learning Strategies in a Design 
Project by Students in Grades 9-12 





Figure 2. Activity and design artifact example in architectural design class 


restrooms, and was required to be handicap accessible. 
Although the architecture challenge was individual, natural 
groupings of students formed and collaboration in terms 
of providing assistance to one another developed. 

Data collection and analysis 

In this study, data were collected through two sources: a 
self-regulated survey for engineering design (EDQ) and a 
Web-based engineering design notebook (WEDN). The 
survey questionnaire was used to capture students' task 
interpretation, perceptions about their planning, cognitive, 
and self-regulating strategies, and information about 
criteria they applied in assessing their own performance. 
Data on task interpretation and use of planning, cognitive 
and self-regulating strategies were also collected through 
the WEDN. Elere students were asked to report their 
activities on projects twice each week using a Moodle- 
based application. Data collected from the design 
notebook were instrumental in triangulating and 
complementing survey data. 


Data from the survey were collected at early, middle, and 
final stages of the design project through an online survey 
tool, in direct reference to the design task assigned by the 
teacher. In the early stage, the survey assessed students' 
task interpretation and planning strategies. In the middle 
stage, the survey assessed their cognitive and 
monitoring/fix-up strategies. In the final stage, the survey 
assessed criteria student used in their judgment of design 
outcomes. The questionnaire was adapted from the 
Inquiry Learning Questionnaire, which had been validated 
in prior research (Butler and Cartier, 2004; Cartier and 
Butler, 2004; see also Nomme et al, 2012). 

Measurement scales of the survey ranged from 1 to 4 
(i.e., 1 = almost never, 2 = sometimes, 3 = often, and 4 
= almost always). The students completed the survey at 
those three different stages of the project in class and it 
was administered by the teacher (see table 1 for a sample 
of the survey items). 


Features 

Examples 

Task Interpretation 

When 1 am asked to work on a design task like the one 1 am about to solve, 1 am being 
asked to get a good overview of the design objectives. 

Planning Strategies 

Before 1 begin to work on the design task, 1 list ways to identify design objectives. 

Cognitive Strategies 

When working on this kind of design task, 1 read the design description (or brief). 

Monitoring and Fix Up 

During my work on my design task, 1 look back at the design description (or brief). 

Criteria for Performance 

At the end of this design task, 1 know that 1 have done a good job when 1 evaluate 
whether a good understanding of the design objectives was achieved. 


Table I. SRL features and examples in the context of defining the design project 
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Dimensions 

# of items 

Cronbach's Alpha 

Task Interpretation 

9 

.80 

Planning Strategies 

9 

.77 

Monitoring & Fix Up 

20 

.91 

Cognitive Strategies 

25 

.91 

Performance Criteria 

9 

.88 


Table 2. Internal reliability scores 


The EDQ was developed and pilot-tested in previous 
research (Lawanto, 2011, Lawanto et al, 2011) to capture 
the relationships among the main features of the SRL 
model for secondary and postsecondary students engaged 
in design project. An exploratory factor analysis was 
conducted to identify the internal reliability of EDQ 
constructs. Table 2 shows that dimensions targeted in this 
study had very high Cronbach's Alpha scores. 

Before analysing data, collected surveys were first 
evaluated for irregularities. There were two suspiciously 
completed surveys that required us to further investigate 
the validity of the responses. As a result of those findings, 
we excluded the two surveys from our data pool and, 
therefore, ended up with 27 surveys to be analysed. The 
analysis process involved evaluating both quantitative and 
qualitative data. Quantitative data collected from the 
survey were analysed by first calculating the mean values 
of the SRL features and then comparing them across the 
two design phases and nine sub-phases. Due to the small 
sample size, non-parametric tests were conducted in this 
study. Second, relationships between SRL features across 
time were represented in graphical views so that student 
SRL profiles across the design process could be visually 
compared. Transitions across items were considered using 
an item mapping analysis that explored the relationship of 
items across SRL features. 

Qualitative data collected from students' design journals 
were first categorised according to the SRL features and 
then coded using Dym and Little's (2009) conceptual 
model. This approach allowed us to identify how SRL 
features were identified within design phases and sub¬ 
phases. Two research assistants segmented and coded the 
quality and SRL features reflected in students' journal 
entries to ensure consistency of segmenting and coding. 
Inter-rater reliability was at acceptable levels (98% 
agreement), and any disagreements between raters were 
reconciled before calculating final frequencies. 


Findings 

Research Question 1: To what degree was students' 
interpretation of the design task reflected in working 
plans and SRL strategies use across the design process? 

To address this question, we first focused attention on the 
design process as a whole. Across time, while engaged in 
their respective design tasks, students reported on their 
engagement in SRL, from interpreting tasks (Tl), to use of 
planning (PS), cognitive (CS) and monitoring/fix-up (MFS) 
strategies, to applying criteria to evaluate their final 
performance (CR). Our initial analyses considered the 
relationships between overall mean scores among these 
different features of self-regulation and cognition. 

From the survey, the findings suggest that, on average, 
students scored higher on task interpretation than on their 
use of strategies. Students were aware of the need to 
identify what they needed to do (/W=3.13, SD=.44). But 
they were less likely to report using strategies for planning 
(M= 2.76, SD=.61), completing design tasks (M= 2.72, 
SD=. 34) or monitoring progress and repairing problems 
(M=2.84, SD=.42). A series of Wilcoxon tests was 
conducted to evaluate whether these gaps between SRL 
features were significant. The results indicated significant 
differences between task interpretation and planning 
strategies (Z=-2.91,p < .01), task interpretation and 
monitoring/fix-up strategies (Z=-2.395, p < .05), and 
cognitive and monitoring/fix-up strategies (Z=-2.223, p < 
.05). This finding was consonant with the reports of self¬ 
regulation evident in student notebooks. Here qualitative 
coding revealed many more entries focused on task- 
interpretation (100 segments), than on considering or using 
planning (63 segments), cognitive (56 segments), or 
monitoring/fix-up (72 segments) strategies. Note that, as 
will be described in more detail below, on the survey, 
students' relatively high sensitivity to task interpretation was 
also reflected in relatively high scores on performance 
criteria, which indicated what they considered to be a good 
design performance. 
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Problem Definition (pdf) 

In a more detailed set of analyses, we analysed data to track 
how SRL unfolded across the problem definition and 
conceptual design phases separately. A detailed report of 
SRL mean scores within each phase, and associated sub¬ 
phases, is provided in figure 3. To aid in analysis, we 
interpreted scores on any SRL feature as low-to-moderate if 
they fell between 1.00 and 2.75 on the four-point scale, and 
as moderate-to-high if they fell between 2.76 and 4.00. 

Findings here suggest that, across sub-phases of Problem 
Definition, students had moderate-to-high awareness of 
design issues required to understand the task (Tl). 
Although students had lower scores on performance 
criteria (CR) than Tl in all three sub-phases of the problem 
definition phase, they nonetheless showed moderate-to- 
high awareness on CR. In particular, students indicated 
that getting a good grasp of the design objectives 
constituted an important criterion in judging design 
success (/W=3.04, SD=. 65). 

However, students' awareness of task requirements 
(reflected in Tl and CR) was not well reflected in planning 
strategies. For example, despite the students' moderate- 


to-high awareness of the need to get a good overview of 
design objectives (M= 3.26, SD=.71), they did not as 
frequently report using planning strategies focused on 
identifying what those design objectives were (A/?=2.37, 
SD=.74). Results of Wilcoxon tests for Problem Definition 
indicated a significant difference between task 
interpretation and planning strategies on design objectives 
(Z=-3.624, p < .01). Although not statistically reliable, 
similar trends were observed for the other two sub-phases 
in problem definition. Specifically, while students showed 
a moderate-to-high awareness of considerations in 
interpreting the task in relation to design metrics and 
constraints (M=3.11 and M= 2.96, respectively), their 
reported engagement in planning strategies in those sub¬ 
phases were at a low-to-moderate level (M= 2.74 and 
M= 2.70, respectively). These findings suggest that 
supporting students to be aware of and engage in 
planning to identify design objectives, measures, and 
constraints could help them to achieve a better, more 
holistic understanding of a design task. 

When considering how students approached clarifying 
design objectives, differences were also observed 
between levels of planning reported (M=2.37, SD=.74) 


All Students (n = 27) 



Features of SRL: Tl: Task Interpretation; PS: 
Planning Strategies; CS: Cognitive Strategies; MFS: 
Monitoring/Fix-Up Strategies; CR: Criteria for 
Performance 


Design Phases and Sub-Phases: (1) Problem 
Definition: dob: clarifying design objectives; dme. 
establishing design metrics; dcon: identifying 
design constraints; (2) Conceptual Design: 
dfunc: Establishing design functions; dreq: 
Establishing design requirements; dmea: 
Establishing design means; dalt: Generating 
design alternatives; dmr: Refining design metrics; 
dsel: Choosing design solutions. 


Figure 3. Comparison of SRL features mean values across design sub-phases for all students 
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and reported use of cognitive and monitoring/fix-up 
strategies (M= 2.64, SD =.56 and/W=2.91, SD=.55, 
respectively). While there were significant differences 
between planning strategies and cognitive strategies 
(Z=-2.559, p < .05) and between planning and 
monitoring strategies (Z=-2.348, p < .05), no significant 
difference was found between cognitive strategies and 
monitoring/fix-up strategies and between task 
interpretation and performance criteria. 

Conceptual Design (cd) 

As we observed during Problem Definition, in the 
Conceptual Design phase students also exhibited 
moderate-to-high awareness related to task interpretation 
(with Tl means ranging from 2.96 to 3.37 across sub¬ 
phases) (see figure 3). In general, students also showed 
moderate-to-high awareness of important criteria for 
judging performance (with CR means ranging from 2.96 
to 3.11), a finding aligned with the students' high Tl. 

In this phase, students also reported moderate-to-high 
levels of planning when establishing design requirements 
(M= 3.04, SD=.81) and means (M= 2.93, SD=. 83), 
generating alternatives (/W=3.04, SD=.81), and choosing 
design solutions (M= 2.78, SD=.97). That said, as during 
Problem Definition, we still observed important gaps 
between students' task interpretation and planning. First, 
while students exhibited moderate-to-high awareness of 
the importance of interpreting tasks to establish design 
functions (M= 3.37, SD=.69), they less frequently 
reported use of good planning strategies to elaborate 
understanding of design goals (M= 2.59, SD=.75). 

Results of Wilcoxon tests for Conceptual Design indicated 
a significant difference between task interpretation and 
planning strategies on design functions (Z=-3.377, p < 
.01). For example, students were well aware of the need 
to understand the action or function for which their design 
must perform, but failed to recognise the need to identify 
the input and output components needed to be 
incorporated into the design. Second, students' scores 
suggested moderate-to-high awareness of the need to 
narrow down and apply a refined set of measures that 
contribute to good design performance (M= 2.96, 
SD=.71), but they appeared to be less aware of the 
benefit of planning to achieve that goal (M= 2.63, 

SD=. 84), as illustrated by their predilection to identify the 
measures before establishing a refined standard 
measurement. While this trend was suggestive, no 
significant difference was found between task 
interpretation and planning strategies on design 
metrics refinement. 


As for the other SRL features, we observed a range in 
students' reported use of cognitive strategies associated 
with Conceptual Design, from a low when identifying 
design functions (M= 2.39, SD=. 59) to a high when 
establishing design means (/W=3.22, SD=. 80). These 
findings suggest that students failed to recognise 
important cognitive strategies applicable in certain sub¬ 
phases. Further, we observed a more consistent lower 
reporting of engagement in monitoring and fix-up 
strategies across most sub-phases, particularly in 
comparison to scores on task interpretation. For example, 
compared to a task interpretation mean of 3.37, when 
establishing design functions students' reported use of 
monitoring and fix-up strategies was much lower (e.g., 

M= 2.65, SD=. 62). Similar gaps between monitoring/fix¬ 
up strategies and task interpretation were found in design 
selection (i.e., /W=2.74, SD=.61; M= 3.26, SD=.81, 
respectively). Results of Wilcoxon tests indicated significant 
differences between task interpretation and monitoring/fix¬ 
up strategies on design functions (Z=-3.108, p < .01) 
and on design selection (Z=-2.458, p < .05). 

Research question 2: How do low- and high-design- 
performing students differ in interpreting and 
deploying SRL strategies? 

Project grades for students were reported by their teacher, 
and were used to cluster students into groups of relatively 
lower and higher performers. Note that, while project 
grades did take into account a group's achievement, they 
also reflected individual differences (e.g., as reflected in 
students' accounts in design notebooks). Overall the 
mean grade across all participating students was 91.07% 
(SD=4.62), which was relatively high. However, we were 
able to identify relatively higher-performing students as 
those whose scores were at least % of an SD above this 
mean (scores > 94.5; n = 12), and relatively lower- 
performing students as those whose scores were at least 
% of an SD below the mean (scores < 87.6; n = 7). By 
employing .75 SD, we could meaningfully differentiate the 
level of design performance and also have enough 
sample size for the two groups. 

To address the research question, we first calculated and 
compared mean scores and significance differences of 
SRL strategies between higher- and lower-performing 
students. Table 3 reports our findings from the survey 
comparing mean scores across the entire design project 
for the two groups of students. What can be observed 
here is that overall mean scores were essentially 
equivalent across groups for task interpretation and 
reported use of cognitive and monitoring-fix up strategies. 
While not statistically reliable, a surprising trend was 
observed from the lower-performing students who 
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SRL Feature 

Higher-performing students 

M (SD) 

Lower-performing students 

M (SD) 

Task interpretation 

3.10 (.44) 

3.03 (.52) 

Planning Strategies 

2.56 (.61) 

2.81 (.48) 

Cognitive Strategies 

2.80 (.34) 

2.71 (.24) 

Monitoring/Fix-up Strategies 

2.96 (.42) 

2.93 (.14) 

Criteria for Performance 

2.96 (.39) 

3.14 (.44) 


Table 3. Comparison of SRL scores for higher- and lower-performing students (N=19) 


reported greater use of planning strategies. Although there 
was no significant SRL difference found between both 
groups of students at the design phase level, Mann- 
Whitney tests revealed significant differences in design 
sub-phases across SRL strategies using one-tailed p-values 
(p < .05). Specifically, we found that higher performing 
students scored significantly higher than their lower- 
performing peer on selecting strategies to establish design 
means and monitoring/fix-up strategies to generate design 
alternatives. We also found lower-performing students 
scored significantly higher on planning their design metrics 
refinement than their high-performing peers. Tentative 
interpretation of this result suggests that, while lower- 
performing students may have been more active in 
planning, they were not more likely to translate those 
plans into action. 

While differences between higher- and lower-performing 
students were not as pronounced in the survey findings, 
analysis of notebook writings did reveal important 
differences across the two groups. First, on average, 
higher-performing students mentioned SRL-related 
features more often in notebooks than did their lower- 
performing peers (i.e., 15 vs. 11 total segments per 
student, respectively). We found from Chi-square test 
conducted that the difference of the total number of 
segments per student between both groups was 
significant (% 2 = 4.154, df = 1, p = .02). Second, higher¬ 
performing students conveyed more detailed and specific 


descriptions of task requirements and strategies enacted 
than did their relatively-lower performing peers. Table 4 
shows the distribution of segment quality between higher- 
and lower-performing students found from their design 
notebooks. Again, Chi-square tests indicated that the 
differences between both groups in terms of the detailed 
and specific journal entries are significant (% 2 = 7.686, df 
= 1, p = .006) and the less detailed and less specific 
entries are also significant (% 2 = 8.000, df = 1, p = .005). 

For example, while both groups may have been relatively 
well aware of the importance of task interpretation 
according to survey results, the qualitative results from the 
notebooks show that higher-performing students 
described their understanding of the tasks in greater detail 
than did lower-performers. Consider, for example, the four 
excerpts below that were drawn from the writing samples 
on task interpretation of higher- and lower-performing 
students, a pair each for robotic and architectural design 
contexts: 

The library also needs to have meeting rooms, 
performance space, computer access area, outside area, 
and office rooms. It must fit in a square corner lots that 
is 150ft x 125ft and be set back 6ft from the property 
line. (Higher-performing Architectural student) 

I understand that 1 have to build a library for a small 
town of 25,000 thousand. (Lower-performing 
Architectural student) 


Groups 

Quality of journal report segment 

Higher-performing students 
(I segments/student) 

Lower-performing students 
(£ segments/student) 

Detailed and specific 

1 15/12 = 9.58 (65%) 

28/7 = 4.86 (37%) 

Less detailed and specific 

61/12 = 5.08 (35%) 

48/7 = 6.86 (63%) 


Table 4. Comparison of journal report guality across groups 
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We will be designing robots that can knock off cans 
from a pedestal, standing up right, and put cans from 
the table on top of the pedestal. (Higher-performing 
Robotic student) 

That we have to pick up cups or knock them off a table, 
and gain points for doing it. (Lower-performing 
Robotic student) 

Thus, although both higher- and lower-performing 
students identified the design requirements for their 
projects, it was clear from the above-referenced excerpts 
that the higher performers were more thorough in 
identifying and describing essential design specifications 
for their projects than were the low performers. 

Similarly, while overall survey mean scores may have been 
similar for monitoring and fix-up strategies across the two 
groups, important differences in the quality of strategy use 
were revealed in the analysis of notebook entries, as 
noted in the following excerpts: 

I mapped out exactly where I want everything to go, 
now I need to think about what they dimensions are 
going to look like. It is going to be some though work, 
but I know I can do it. I think I need to improve on 
making the window dimensions more accurate. (Higher¬ 
performing Architectural student) 

I am struggling with find the right measurement for each 
of the rooms. (Lower-performing Architectural student) 

Today we finalised the elevator for our robot and we 
figured out a way to put the string on the robot, but our 
claw design is pretty flawed and is very flimsy. (Higher¬ 
performing Robotic student) 

Got farther on the design and built of the robot. (Lower- 
performing Robotics student) 

But, while we found group differences in the quality of 
self-regulation between lower- and higher-achieving 
students, as described above, survey data revealed similar 
gaps between task interpretation and reported use of 
strategies for both groups of students. Inspection of 
means suggested that higher achievers were relatively 
high on task interpretation, compared to lower scores for 
planning, cognitive and monitoring/fix-up strategies (see 
table 3); however, significant differences (p <.05) were 
found only between task interpretation and planning and 
task interpretation and cognitive strategies. A similar trend 
was apparent for lower-achieving students, where task 
interpretation was significantly higher than were reports of 
planning and use of cognitive strategies (p <.05). 


Similarly, notebook data showed that, consistent with 
earlier analyses, both higher- and lower-performing 
students focused more on task interpretation than they 
did on describing and implementing strategies. Higher- 
performers described task-interpretation-related issues 
more often (62 segments) than they described issues 
related to planning (28 segments), cognitive (46 
segments), or monitoring and fix-up (40 segments) 
strategies. Similarly, low-performing students' focus on task 
interpretation (23 segments) outstripped their mentioning 
planning (18 segments), cognitive (16 segments), or 
monitoring/fix-up-related issues (16 segments). 

Conclusions and discussion 
Conclusions 

Our goals in conducting this exploratory study were to 
develop a methodological framework for evaluating 
students' SRL in an engineering design activity and to 
identify patterns of students' SRL while solving a design 
problem. From the findings, we can conclude that the 
grade 9-12 students in this study placed more attention 
on understanding of task requirements than developing 
proper plans, selecting strategic actions to implement the 
plans, and monitoring activity in order to solve the design 
task. Gaps between students' interpreting task demands 
and selecting strategies for task completion and 
monitoring/self-adjusting approaches were apparent for 
both higher- and lower-achieving students at this level. 

The relatively high scores on students' understandings of 
tasks were also paralleled in their high scores on 
performance criteria. 

Even for this high-performing sample of students, 
important differences in self-regulation were observed 
between relatively higher- and lower-achieving students. 
When looking across the two design phases studied 
(problem definition; conceptual design), overall mean 
comparisons on the EDQ revealed no statistically reliable 
group differences. However, some important differences 
were observed at the sub-phase level. For example, lower- 
performing students scored significantly higher on 
planning their design metrics refinement than did their 
high-performing peers. In contrast, higher performing 
students scored significantly higher than did lower- 
performing peers on selecting strategies to establish 
design means and monitoring and fix-up strategies to 
generate design alternatives. These findings combined to 
suggest that, even when lower-performing students may 
have been more active in planning, they were less likely 
than were high-achieving peers to translate those plans 
into action. Further, striking group differences across all 
SRL-features were revealed in notebook writings. Students 
who received the highest grades in their Engineering 
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design courses also provided greater depth and specificity 
in their descriptions of task requirements and strategies 
used to achieve them. 

Discussion 

The first general principle for K-12 engineering education 
set by the Committee on K-12 Engineering Education is to 
emphasise engineering design in K-12 engineering 
education (Katehi et al, 2009). Despite a strong push to 
emphasise design in K-12 engineering education, research 
has found students, particularly in grades 9-12, to be less 
skillful in deploying good metacognitive skills in learning 
(Joseph, 2010; Lawanto, 2011,2012; Mateos et al, 
2008). Gaps between understanding task requirements 
and developing proper plans, selecting strategic actions to 
implement the plans, and monitoring design activity may 
hinder students in solving design problems more 
systematically. Thus, our findings here suggest that 
teachers should devote increased time to in-depth 
discussion of the design process and SRL strategies that 
may be critical to solving tasks in each phase throughout 
that design process. For example, teachers may allocate 
more time to support the students in transforming their 
understanding of the tasks into proper planning strategies. 
Also, encouraging students to discuss with their 
teammates about their understanding of what they are 
being asked to do in the design project may help them 
refine their interpretation of the tasks being asked of them. 
Teachers may offer their help in this process by 
encouraging the students to review their understanding of 
the task and to continually ask for help and clarification. 

Differences between higher- and lower-performing 
students in their task interpretation and strategy use might 
be triggered by other issues such as emotions, task value, 
and creativity. Students with positive emotions while 
engaged in design task can persistently execute their 
strategies to achieve their goals (Pekrun et al, 2011). 
Similarly, students who express positive task value may 
perform better than those who see the activity as an 
unimportant, irrelevant, or uninteresting (Bong, 2001). 
Furthermore, creativity, which is considered as an essential 
factor for generating ideas (Barak, 2007), may also be 
able to explain why higher-performing students are less 
likely to report their planning strategies, but often execute 
relevant cognitive strategies well. Since creative students 
may dynamically change their design strategies, they may 
feel that is not too essential to report their planning 
strategies. In essence, some planning strategies become 
tacit knowledge (Polanyi, 1962). In contrast, students who 
are less creative may exhibit the ability to explicitly report 
their plans required to achieve their goals. 


Since many of our findings were suggestive and this study 
was exploratory in nature, interpretation of the data should 
be done with caution and future work in this area is highly 
recommended. Three directions are proposed for the 
improvement of future work. First, studies with larger 
sample sizes are essential so that the generalisability of 
the findings can be determined. A larger sample size 
would add statistical power in evaluating differences 
between SRL features and across groups of students. 
Sampling from multiple schools, with learners at a larger 
range of achievement levels, may also assist in 
understanding individual differences in SRL as they unfold 
within a diversity of contexts. Second, future research 
should investigate connections between how engineering 
design activities are introduced and supported at Grade 
9-12 levels and how students think about and engage in 
design processes. Having a better understanding of these 
connections may further help us in targeting how to shape 
classroom practices that enable self-regulation for 
secondary school students. Third, the second and third 
SRL features of Butler and Cartier's model (i.e., what 
individuals bring into context and mediating variables) 
should be included in our future works. Although the data 
analysis may become more complex, creativity, emotions, 
and task value may open doors to a better understanding 
of why students tend to have lower planning strategies 
when measured against their understanding of the design 
task and how higher- and lower-preforming students differ. 
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